1. Field of Invention
The following relates to image resizing using content-aware image retargeting.
2. Description of Related Art
Due to the fast growing diversity of display devices, an image often needs to be displayed across various imaging devices having different image settings, including different aspect ratios. To assure a good viewing experience in a given display device, it may be necessary to resize, or otherwise modify, an original image for a given display device.
FIG. 1 illustrates four methods of modifying an original image 11 for display on a screen 13 that has an aspect ratio different from that of original image 11. One method of displaying original image 11 on screen 13 is to reduce the size of image 11 to fit entirely within screen 13. The shrinkage of original image 11 however degrades the viewable enjoyment of the image, particularly if screen 13 is small. Additionally, this approach does not take advantage of the entire viewable area of screen 13, as is evident by the introduction of blank spaces, i.e., black bars 2 and 4 above and below the displayed image, respectively.
Another option is to compress original image 11 to create a compressed representation 11A that matches the aspect ratio of the display screen 13. This results in a distorted image 13a being displayed. As seen, displayed image 13a experiences an elongated, stretching quality, as compared with original image 11.
A further option is to crop original image 11 to fit display screen 13 by using a cropping frame 15 whose aspect ratio matches that of display screen 13. But cropping results in a displayed image 13B that may omit important image content from original image 11. For example in the present case, in order to include the person standing off from the castle in original image 11, it is necessary to cut off half of the castle in displayed image 13B.
A more recently proposed option is termed content-aware image retargeting, and it aims to remove from view, or otherwise compress, less important image content information while preserving, or otherwise minimizing compression/distortion of, more important image content information. A simplified illustration of this idea is to divide original image 11 into a multiple image segments S1 through Sn.
In the present example, image segments S1 through Sn are shown large, wide, and vertical, but this is purely for illustration purposes. More typically, each image segment, or seam, is one pixel wide and follows a path that goes straight or shifts by one-pixel position as it crosses from one pixel row/column to the next in traversing from one side of the image to its opposite side. Preferably, each pixel in the seam's path is selected according to image content importance so that each seam defines a path of least importance as it progresses from one row/column to the next. Thus, the shifting of the seam is the result of the pixel paths identifying the pixels of least importance, i.e., pixels that are part of less important image content.
Content-aware image retargeting may automatically remove seams to reduce an image size, or insert seams to extend it, as necessary, for a given display screen. In the present simplified illustration, the resultant, displayed image 13C preserves the entire castle image content intact as well as the person standing off from the castle, but removes (or optionally compresses) the portion of original image 11 that contains less important image content information, such as the sky and field space between the person and the castle.
An objective of content-aware image retargeting is to preserve the look-and-feel (i.e., visual quality) of the original image. One method of achieving this is to maintain consistency of important structures from the original image to the resized output image (i.e., the target image). However, compression of less important image content often introduces distortion into the structure of more important content. For example, if less important content that surrounds (or is adjacent to) more important content is removed, or compressed; this may introduce curvature into the straight lines of the more important content.
Nonetheless because of its ability to preserve important imaging content, content-aware image retargeting has attracted much interest recently. A challenging issue, however, is how to balance a tradeoff between preserving important content, minimizing visual distortion, and maintaining consistency of an image structure.
It is an object of the present invention to provide an improved method of identifying important content for use in content-aware image retargeting.
Another object of the present invention is to preserve important content of an image while minimizing visual distortion and maintaining consistency of from an original image.